Skip to content

Computing crate_hash from metadata encoding instead of HIR (implements #94878) (very draft)#154724

Draft
Daniel-B-Smith wants to merge 3 commits intorust-lang:mainfrom
Daniel-B-Smith:smithdb3/fix-94878
Draft

Computing crate_hash from metadata encoding instead of HIR (implements #94878) (very draft)#154724
Daniel-B-Smith wants to merge 3 commits intorust-lang:mainfrom
Daniel-B-Smith:smithdb3/fix-94878

Conversation

@Daniel-B-Smith
Copy link
Copy Markdown
Contributor

@Daniel-B-Smith Daniel-B-Smith commented Apr 2, 2026 •
edited
Loading

View all comments

Adds a parallel process of hashing along with the metadata encoding. It does add metadata encoding in a few places to ensure that the hash was available. Currently, the metadata encoding always generates both the metadata and the hash even if only one is needed.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 2, 2026
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Daniel-B-Smith Daniel-B-Smith force-pushed the smithdb3/fix-94878 branch 5 times, most recently from 24c04d5 to c826667 Compare April 7, 2026 15:59
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Daniel-B-Smith Daniel-B-Smith force-pushed the smithdb3/fix-94878 branch 2 times, most recently from 43a7704 to 338aba3 Compare April 8, 2026 19:34
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Daniel-B-Smith Daniel-B-Smith force-pushed the smithdb3/fix-94878 branch 2 times, most recently from 0ddcd96 to d27cca5 Compare April 8, 2026 21:34
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Daniel-B-Smith Daniel-B-Smith force-pushed the smithdb3/fix-94878 branch 3 times, most recently from 1e5f269 to 4171895 Compare April 9, 2026 14:56
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-bors

This comment has been minimized.

Sign in to view
@Daniel-B-Smith Daniel-B-Smith changed the title #94878 Computing crate_hash from metadata encoding instead of HIR (implements #94878) (very draft) Apr 9, 2026
@nnethercote
Copy link
Copy Markdown
Contributor

nnethercote commented Apr 9, 2026

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rust-bors

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 9, 2026
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 1, 2026
@rust-bors

This comment has been minimized.

Sign in to view
rust-bors Bot pushed a commit that referenced this pull request May 1, 2026
@bors
Auto merge of #154724 - Daniel-B-Smith:smithdb3/fix-94878, r=<try>
9e15878 
Computing crate_hash from metadata encoding instead of HIR (implements #94878) (very draft)
@rust-bors
Copy link
Copy Markdown
Contributor

rust-bors Bot commented May 2, 2026

💔 Test for 9e15878 failed: CI

@rust-bors

This comment has been minimized.

Sign in to view
@JonathanBrouwer
Copy link
Copy Markdown
Contributor

JonathanBrouwer commented May 4, 2026

@bors try

@rust-bors

This comment has been minimized.

Sign in to view
rust-bors Bot pushed a commit that referenced this pull request May 4, 2026
@bors
Auto merge of #154724 - Daniel-B-Smith:smithdb3/fix-94878, r=<try>
ad26d1a 
Computing crate_hash from metadata encoding instead of HIR (implements #94878) (very draft)
@rust-bors
Copy link
Copy Markdown
Contributor

rust-bors Bot commented May 4, 2026

☀️ Try build successful (CI)
Build commit: ad26d1a (ad26d1abc0cd7a7ea9b98cf477d0f887f415fd55, parent: 1d72d7e8136faaebad3a85eeed432e6ea1b2ffab)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Copy Markdown
Collaborator

rust-timer commented May 4, 2026

Finished benchmarking commit (ad26d1a): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)		 0.3% [0.1%, 0.6%] 17
Regressions ❌
(secondary)		 0.2% [0.1%, 0.4%] 16
Improvements ✅
(primary)		 -0.6% [-5.2%, -0.2%] 33
Improvements ✅
(secondary)		 -0.6% [-1.9%, -0.2%] 27
All ❌✅ (primary) -0.3% [-5.2%, 0.6%] 50

Max RSS (memory usage)

Results (primary -4.1%, secondary 1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 1.9% [0.4%, 7.6%] 6
Improvements ✅
(primary)		 -4.1% [-4.1%, -4.1%] 1
Improvements ✅
(secondary)		 -2.4% [-2.4%, -2.4%] 1
All ❌✅ (primary) -4.1% [-4.1%, -4.1%] 1

Cycles

Results (primary -7.2%, secondary -5.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 0.7% [0.6%, 0.8%] 3
Improvements ✅
(primary)		 -7.2% [-7.2%, -7.2%] 1
Improvements ✅
(secondary)		 -9.1% [-22.2%, -0.5%] 5
All ❌✅ (primary) -7.2% [-7.2%, -7.2%] 1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 494.198s -> 496.007s (0.37%)
Artifact size: 394.42 MiB -> 394.46 MiB (0.01%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 4, 2026
@rustbot rustbot added the WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver) label May 5, 2026
@Daniel-B-Smith
Copy link
Copy Markdown
Contributor Author

Daniel-B-Smith commented May 5, 2026

As somewhat expected, restoring the hir_crate hash made all of the incr-unchanged profiles into wins but converted the incr-patched cases into regressions. The latter regressions are smaller than the regressions with hir_crate no_hash.

Since the primary regression with no_hash is dependency analysis being re-run, it seems unlikely that those regressions can be substantially improved. Conversely, the HIR hash should be able to be made less expensive since it is no longer quite as load bearing.

The next change is to make the HIR hash cheaper by removing components where the dependency check is handled elsewhere and using a commutative combine on the remaining fingerprints. A number of inputs to !tcx.needs_metadata() were removed as well since those no longer go into the SVH.

All of tests/incremental is passing, but I'll wait for a full run for at least one architecture before asking for a perf bot since this change is probably riskier.

@mejrs
Copy link
Copy Markdown
Contributor

mejrs commented May 5, 2026

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 5, 2026
@rust-bors

This comment has been minimized.

Sign in to view
rust-bors Bot pushed a commit that referenced this pull request May 5, 2026
@bors
Auto merge of #154724 - Daniel-B-Smith:smithdb3/fix-94878, r=<try>
9180a5c 
Computing crate_hash from metadata encoding instead of HIR (implements #94878) (very draft)
@rust-bors
Copy link
Copy Markdown
Contributor

rust-bors Bot commented May 5, 2026

☀️ Try build successful (CI)
Build commit: 9180a5c (9180a5c6a9ad54fd478c2f21273a551c73f21cc9, parent: 740679e1f5554a3c1695317e084809e3fceb9825)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Copy Markdown
Collaborator

rust-timer commented May 5, 2026

Finished benchmarking commit (9180a5c): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)		 0.4% [0.2%, 0.5%] 6
Regressions ❌
(secondary)		 0.5% [0.1%, 1.0%] 12
Improvements ✅
(primary)		 -0.5% [-5.5%, -0.1%] 68
Improvements ✅
(secondary)		 -1.4% [-5.0%, -0.1%] 46
All ❌✅ (primary) -0.5% [-5.5%, 0.5%] 74

Max RSS (memory usage)

Results (primary -2.9%, secondary -1.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 0.7% [0.6%, 1.1%] 4
Improvements ✅
(primary)		 -2.9% [-4.3%, -2.2%] 3
Improvements ✅
(secondary)		 -1.9% [-8.2%, -0.5%] 11
All ❌✅ (primary) -2.9% [-4.3%, -2.2%] 3

Cycles

Results (primary -4.8%, secondary 1.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 10.4% [1.1%, 24.8%] 6
Improvements ✅
(primary)		 -4.8% [-8.7%, -2.5%] 3
Improvements ✅
(secondary)		 -2.7% [-5.8%, -0.4%] 15
All ❌✅ (primary) -4.8% [-8.7%, -2.5%] 3

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 495.494s -> 494.288s (-0.24%)
Artifact size: 394.42 MiB -> 394.47 MiB (0.01%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) perf-regression Performance regression. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@Daniel-B-Smith @rust-log-analyzer @nnethercote @rust-timer @Zalathar @mejrs @Kobzol @ZuseZ4 @JonathanBrouwer @rustbot